3. RATIFY EMERGENCY EXPENDITURE OF FUNDS APPROVED BY GENERAL MANAGER FOR DATA RECOVERY
Meeting
Date: February 19, 2004 Budgeted: No
Program Line Item No.: Contingency
Inder M
Osahan
General
Counsel Approval: N/A
Committee
Recommendation: The Administrative Committee reviewed this item on February 10,
2004 and recommended approval.
CEQA
Compliance: N/A
SUMMARY: The
District suffered a hardware failure on January 26, 2004. Staff worked with
tools installed and available on-site, along with equipment manufacturers, to
restore the data files stored on the system but was unable to do so. Due to the severity of the failure, data stored
on the damaged hard disk drives was recovered by retaining services of Ontrack
Data Recovery, Inc. Ontrack recovered
all of the lost data and staff has been able to conduct business as usual since
that time. Utilizing the authority
granted in Resolution No. 2002-08, which increases the General Manager’s
authority to approve contracts from $2,000 to $20,000 in emergency situations,
the General Manager approved the expenditure of $12,514 for this purpose.
RECOMMENDATION: Staff
recommends Board ratification of the expenditure of contingency funds in the
amount of $12,514 to Ontrack Data Recovery, Inc. The Administrative Committee considered this matter at their
February 10, 2004 meeting and voted 3 to 0 to recommend approval. The Committee also recommended that staff
attempt to recoup the cost from the manufacturer of the failed equipment.
BACKGROUND: On
Monday, January 26, 2004, the District’s electronic file storage server
suffered a severe hardware failure.
Files stored on the system were inaccessible for the day while staff
worked with the equipment manufacturer to try to restore the system. In addition to the hard disk failure, the
current backup system for the District had also failed due to faulty hardware
that went undetected by the backup system.
The data restore process was unsuccessful and the District retained
Ontrack Data Recovery, Inc. to help restore the data. The data recovered by Ontrack has been restored to the production
servers and is available to staff to conduct regular business. The data recovery was 100 percent successful
and has saved considerable staff time and resources. Information gathered from eight staff members indicated that over
$30,000 in staff time would have had to be expended just to recreate their own data. It is estimated that the cost of time
expended by the entire staff would have totaled almost $100,000.
During
the time the regular server was unavailable, staff deployed a standby server to
conduct business as usual while the data was being recovered from the damaged
hard disk drives. To avoid having a
reoccurrence of a disaster of this magnitude, staff has implemented redundancy
not only for the file server, but also for system monitoring and backups.
U:\staff\word\boardpacket\2004\2004boardpacket\20040219\ConsentCalendar\3\item3.doc